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Abstract. This paper studies tiie brancliing time equivalences and pre- 
orders for continuous-time Markov decision processes (CTMDP), and tiie 
logical characterization problem of these relations with respect to the 
continuous-time stochastic logic (CSL). For strong bisimulation, it is well 
known that bisimulation is strictly finer than CSL equivalence. In this pa- 
per, we propose the notion of weak bisimulations for CTMDPs and show 
that for a subclass of CTMDPs, weak bisimulation is both sound and 
complete with respect to the equivalence induced by the sub-logic of CSL 
without next operator. We then propose a sequence of i-depth bisimula- 
tion relations characterizing a sequence of sub-logics with bounded until. 
The i-depth bisimulation equivalences converge to the CSL equivalence 
for arbitrary CTMDPs. Further, we extend the framework to simulations 
and their characterizations as well. Another notable contribution of the 
paper is the notion of a parallel composition operator for CTMDPs, more- 
over, we show that both strong and weak bisimulations are congruence 
relations with respect to it. 



1 Introduction 

Recently, continuous-time Markov decision processes (CTMDP) have received 
extensive attentions in the model checking community, see for example [2, 20, 
21,4, 22, 6, 5, 23, 14]. Analysis techniques for CTMDPs suffer especially from the 
infamous state space explosion problem. Thus, as for other stochastic models, 
strong bisimulations have been proposed for CTMDPs in [20], which were shown 
to be sound with respect to the continuous-time stochastic logic (CSL). This 
result guarantees that one can first reduce the CTMDP using bisimulations be- 
fore analyzing the CTMDPs, as in the standard setting. On the other hand, as 
indicated in the paper [20], strong bisimulation is not complete with respect to 
CSL, i.e., logically equivalent states might not be bisimilar. 

CTMDPs can be considered as extending the Markov decision processes 
(MDPs) with exponentially sojourn time distributions, and this subsumes models 
such as labeled transition systems and Markov chains as well. While linear and 
branching time equivalences and preorders are studied for these submodels [29, 
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30, 3, 27], this has not been studied for CTMDPs. This paper studies the branch- 
ing time equivalences and preorders for CTMDP, and the fogical characterization 
problem of these relations with respect to the CSL. 

We start with a slightly coarser notion of strong bisimulations, and then pro- 
pose the notion of weak bisimulations for CTMDPs. We study the relationship 
between weak bisimulations and the logical equivalence induced by CSL\x, the 
sub- logic of CSL without the next operator. Our first contribution is to iden- 
tify a subclass of CTMDPs under which our weak bisimulation coincides with 
CSLyx equivalence. We discuss then how this class of CTMDPs can be efficiently 
determined. 

Recently, in [27], we have introduced a sequence of i-depth bisimulations, 
which are shown to be converging to the logical equivalence with respect to 
probabilistic CTL (PCTL). As a second part of this paper, we propose strong 
and weak i-depth bisimulations for CTMDPs, and provide logical characteriza- 
tion results for them. We show that, for general CTMDPs with finitely many 
states, the strong and weak i-depth bisimulations converge to equivalence rela- 
tions which are exactly the CSL and CSLyx equivalences, respectively. 

Further, we extend the definitions to (weak) simulations, and study their 
relationship to the logical preorders with respect to the (weak) safety CSL re- 
spectively. As CTMDPs can be considered as combining MDPs and CTMCs, we 
will discuss the downward compatibility of the relations with those for MDPs 
and CTMCs. 

As another notable contribution, we propose a novel - and very simple - 
parallel composition operator for CTMDPs. We show that both strong and weak 
bisimulations are congruence relations with respect to this new operator. As a di- 
rect consequence of this result, (weak) bisimulation compositional minimization 
reduction technique can be applied for analyzing the CTMDPs. 

Summarizing, this paper introduces various (weak) simulation and bisimula- 
tion relations, and develops for the first time a taxonomy of logical characteri- 
zations of these relations on CTMDPs: 

— We introduce a new notion of weak bisimulation for CTMDPs. We identify a 
subclass of CTMDPs and show the sound and complete characterization for 
CSL\x. 

— We present a sequence of i-depth (weak) bisimulations and the corresponding 
logical characterization results. 

~ We extends the definitions and logical characterization results to (weak) 
simulations and i-depth (weak) simulations. 

— We introduce a novel parallel operator for CTMDPs, and study the congru- 
ence property of strong and weak bisimulations and simulations with respect 
to it. 

Organization of the paper. Section 2 recalls the definition of CTMDPs and the 
logic CSL. In Section 3 we propose a parallel composition operator for CTMDPs. 
Strong and weak bisimulation relations and the corresponding logical character- 
ization results are studied in Section 4. In Section 5, we present the sequence 
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of i-dcpth bisimulations. The work is extended to simulations in Section 6. In 
Section 7, we discuss how the (bi)-simulations on CTMDPs relate to those in 
probabilistic automata and Markov chains. Related work is discussed in Section 
8. Section 9 summarizes the paper and concludes the paper. 

2 Preliminaries 

For a finite set S, a distribution is a function : 5* — > [0,1] satisfying := 
'l2ses ^^(^) — 1- denote by Dist{S) the set of distributions over S. We shall 
use s,r,t, . . . and /i, . . . to range over S and Dist{S), respectively. The support 
of fj, is defined by Supp{^) ~ {s & S \ n{s) > 0}. For an equivalence relation 
72. over S, we write /i 72. if it holds that /i(C) = for all equivalence 

classes C € S/TZ. A distribution /i is called Dirac if \Supp{^)\ = 1, and we 
let I?s denote the Dirac distribution with 2?s(s) = 1. Given two distributions 
Hi and fj.2 such that + |/i2| < l, then jii + /i2 is a distribution such that 
(/zi +fJ.2)is) = /ii(s) + ^2(s) for each s € S. Let /x — C be a distribution such that 
{fj, — C){s) — fi{s) if s ^ C, otherwise (/x — C)(s) = 0, where CCS*. Moreover 
X ■ H with X ■ < 1 is a distribution such that (a; • /i)(s) ~ x ■ fi(s) for each 
s G S. 

Let 72 be a relation over S, define 7^(C) = {r | s 72 r As £ C} and 72"i(C) = 
{r \ r TZ s A s £ C}. We say C is 72 upward closed iff C = 72(C), and similarly 
C is 72 downward closed iff C = 72^^ (C). 

2.1 Continuous-time Markov Decision Process. 

Definition 1. A tuple C = {S, AP, L, sq) is a continuous-time Markov de- 
cision process ('CTMDPj where S is a finite but non-empty set of states, — >C 
S X i?+ X Dist{S) is a finite transition relation where is the set of positive 
real numbers, AP is a finite set of atomic propositions, L : S ^ 2^^ is labeling 
function, and sq £ S is the initial state. M 

We also write s ^ ^ if (s,A, ^) S— Let Suc{s) = {r \ 3{s ^ ii).fj,{r) > 0} 
denote the successor states of s, and Suc*{s) the transitive closure. To avoid 
timelock, we assume w.l.o.g. that Suc{s) ^ for each s £ S. A state s is said to 
be absorbing, denoted as s±, iff V(s' G Sue* {s)).L{s') — L{s). 

A continuous-time Markov chain (CTMC) is a CTMDP satisfying the condi- 
tion that: s A> /i and s h' imply A = A' and /x = /i'. 

Below we recall the notion of uniformization for CTMDPs [6, 21]. Essentially, 
by uniformizing each state will have a unique exit rate while preserving certain 
properties. 

Definition 2. Given a CTMDP C = {S, AP, L, sq), the uniformized CTMDP 
is denoted as C = {S, , AP, L, sq) such that 5 = {s | s G S}, L{s) = L{s) for 
each s € S and (s, E, jl) G— >' iff there exists (s, A, /i) G— > and p. ^ ■ jj' + {1 — 
^) • Vg where fJ,'{f) — iiir) for each r G Supp{iJ.) and E = max{A | (s. A, /i) G— >} 
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is the maximum rate in the original CTMDP. A CTMDP C is uniformized iff for 



2.2 Path and Measurable Scheduler 

Let C = (S*, ^, AP, L, So) be a given CTMDP. Let Paths'^+^C) = S x {R+ x 5)" 
denote the set of paths with length n + 1 of C. The set of all the finite paths of C is 
the union of all the Paths"{C) with n > 0, that is, Paths*{C) = Un>o Paths'^ {C). 
In addition Paths°°{C) = S x {R+ x S)°° contains all the infinite paths and 
Paths {C) = Paths* (C) U Paths°°{C) is the set of ah the paths of C. Intuitively, 
a path is comprised of alternation of state and its sojourn time. To simplify the 
discussion we introduce some notations. Given a path lu — sq, to, si,ti ■ ■ ■ Sn~i G 
Paths^{C), = n is the length of lo, oj s„_i is the last state of w, uj\^ = 
So, tf), - ■ ■ ,Si is the prefix of ui ending at the i-th state, and uj\i ~ Si, ti, s^+i, ■ • • is 
the suffix of a; starting from the i-th state, and uj^(t„-i, s„) is the path obtained 
by extending w with i„_i,s„. Let uj[i] = Si and time{uj,i) = ti denote the i-th 
state and the time spent in the i-th state respectively where i < n, while w@t 
is the state at time point t in to, that is, ui@t = u![j] where j is the smallest 
index such that J2i=o^i ^ ^- Moreover, Steps{s) = {{\, ii) \ {s,X,fi) £— >■} is 
the set of all available choices in state s. Let {li C [0, oo)}o<i<fc denote a set of 
intervals, then C(so, Iq, - ■ ■ , Ik-i, Sk) is the cylinder set of paths w G Paths°° (C) 
such that uj[i] = Si and time{uj,i) £ It. Let dpaths=°{C) be the smallest a algebra 
on Paths°°{C) containing all the cylinder sets. Refer to [20] for more details. 

Non-deterministic choices in CTMDPs are resolved by schedulers, which gen- 
erates a distribution over the available transitions based on the existing path. We 
consider measurable timed history-dependent randomized schedulers [31,20]. 

Definition 3. A scheduler tt : Paths* (C) x i?+ x Dist{S) [0,1] is measurable if 
7r(w, A,^) > implies (A,/i) G Dist{Steps{uj D) and 7r(-, tr) : Paths* (C) i-t- [0, 1] 
are measurable for all tr e 2^^ xDtst{s)) ^ g 

Given a scheduler tt a unique probability measure Pr^^^^g can be defined on the 
C7 algebra 5'pat;is~(C) by: Pr^^soiC{so)) = 1 and Pr^^^o (C(so, 4, • • • ,Sn,In, s„+i)) 
equals: 



any (si, Ai,^i) £ 



and (s2, A2,^2) Ai = A2. 




•(e 



— A-a 



— e 




where /„ = [a,b] 



2.3 Continuous Stochastic Logic 



Continuous stochastic logic (CSL) is introduced to reason about continuous-time 
Markov chains [1], and to reason about CTMDP later on in [20]. It contains both 
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state^ and path formulas whose syntax is defined by the following BNFs: 

LP :■= a\^Lp\Lp Alp\ V^piip) 

where a G AP, p £ [0, f], / C [0, oo) is a non-empty closed interval and txi G 
{<, <, >, >}. We also introduce a bounded until operator ip Ufj (p, a restricted 
version of the general until operator (pU^ ip, which bounds the length of the 
paths we should consider. 

We use s 1= (y9 to denote that s satisfies the state formula (p while lj \^ ijj 
denotes that lu satisfies the path formula ^p. The satisfaction relation for atomic 
proposition and boolean operators are standard. Below we give the satisfaction 
relation for the remaining state and path formulas: 

s ^ Vr^pii}) iff V7r./'r^.,„({w e Paths°" \ uj ^ tl;}) txi p 
w 1= iff uj[l] ^ ipA time{LO, 0) e I 
uj^ (piU' ip2 iff 3t e I.{uj@t 1= 1(92 A (Vt' < t.uj@t' 1= (pi)) 
Lj ^ ipiU'n(p2 iS 3i < n At e I.{Lj@t = Lj[i] 

A ujm h "^2 A (Vi' < t.uj@t' h <^i)) 

Logic Equivalences. We say s and r be CSL-equivalent, denoted by s ^csl 
if they satisfy the same set of formulas of CSL, that is, s \= (p iS r \= (p for all 
state formulas (p. Similarly for sub-logics of CSL. In the following, we let 

— CSL~ denote the sub-logic of CSL without unbounded until operator, 

— CSL\u„ denote the sub-logic without bounded until, 

— CSL\x denote the sub- logic without next and bounded until, and 

— CSLi be the sub- logic such that all the bounded until operators are like 
(pi ip2 with j < i. 

The subscripts i.e. — , X, U„, and i can be applied to CSL at the same time (for 
instance CSL^). 

3 Parallel composition for CTMDPs 

Compositional theory plays an extremely important role in verification, as com- 
position based minimization and verification are effective methods for solving the 
state space problem. For all sub-models of CTMDPs, including CTMCs and prob- 
abilistic automata, their compositional theories have been studied extensively in 
the literature [15,24,9,13]. Surprisingly, to the best of our knowledge, the par- 
allel operator has not been defined for CTMDPs. Indeed, thus far, CTMDPs are 
considered as non-compositional. In this section, we define a novel parallel com- 
position operator for CTMDPs - directly inspired by the parallel composition for 
CTMCs [15]. We will show that the strong and weak bisimulations we introduce 
are compositional with respect to our parallel composition. 

^ The steady-state operator is omitted in tills paper for simplicity of presentation. 
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Fig. 1. Parallel composition of sq and to (transitions out of other states are omitted, 
only transitions out of so,to, so||to are depicted). 



Definition 4. Let d = {Si, APi, Li, Si) with i ~ 1,2 be two CTMDPs, and 

Ml II be a distribution such that (fii \\ /^2)(si || S2) = /^i(si)-/^2(s2)- The parallel 
composition Ci || C2 is defined by: Ci || C2 = {Si || S2, — >, APi x AP2, i, si || S2) 
where 

" 5i||52 = {s||s'|sg5iAs'g52}, 

- L{s \\s') ~ L{s) X L{s'), and 

— {s\\s',X,^) whenever there exists {s,Xi,^i) G^i and (s',A2,/X2) G— 7>2 
s.t. A = Ai + A2 and fi^ ^- {pi \\V,,) + ^ ■ {V, \\fi2). 

The following example illustrates how the composition operator works. 

Example L Given two processes so and to as in Fig. 1 (a) and (b) respectively, 
where sq has two non-deterministic transitions labeled with 2 and 3, and to only 
has one transition with rate 4, then the parallel composition so 1 1 ^o of sq and t^ 
according to Definition 4 is described as in Fig. 1 (c). 



Discussion. The parallel composition is inspired by the parallel operator in- 
troduced for CTMCs in [15]. The extension is conservative, i.e., restricting to 
CTMCs, our parallel composition agrees with that for CTMCs. The parallel op- 
erator have been extended for both interactive Markov chains (IMCs) in [15] and 
Markov Automata (MAs) [13]. In both IMCs and MAs, for each state at most 

one transition is labeled with Markovian rate s \ fi, that is, no nondetermin- 
istic choices between Markovian transitions are allowed. But nondeterministic 
choices between transitions labeled with actions are allowed. With our notion of 
parallel compositions for CTMDPs, we believe that a compositional theory for 
stochastic behavioral models without restricting nondeterministic choices among 
Markovian transitions can be studied. 
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4 Bisimulations for CTMDPs 
4.1 Strong Bisimulation 

In this section we recall the notion of strong bisimulation for CTMDPs, intro- 
duced in [20], where s A-p fi iff there exists {s \ and {pi}i^i such that 
Pi e (0, 1] for each i £ /, J2iei Pi ~ ^.nd X^ie/ Pi-f^i — M- We assume that there 
is a given CTMDP C = {S, — AP, L, sq) throughout the paper in the following. 

Definition 5. Let TZ ^ S x S be an equivalence relation. TZ is a strong bisimula- 
tion if s TZ r implies that L{s) = L(r) and for each s A- ^, there exists r A-p fi' 
such that ^ TZ fj.' . We write s ~ r whenever there exists a strong bisimulation 
TZ such that s TZ r. M 

The above bisimulation relation is slightly coarser than the one considered 
in [20], where r Ap fi' is replaced by strong transition r \ fi'. The idea of 
combining transitions with the same exit rate is borrowed from [25] . The theorem 
shows that strong bisimulation is sound, but not complete for CSL equivalence: 

Theorem 1 ([20]). - C --csl- ■ 

Note the proof in [20] can be directly adapted to our slightly more general strong 
bisimulation. The inclusion is sound but not complete which is illustrated below: 

Example 2. Suppose that wc have two states s and r such that s can evolve into 
si cither with rate 3 or 5 while r can evolve into si with rate 3, 4, or 5. Also we 
assume that L{s) ~ L(r) and si is an absorbing state with L{si) ^ L(s). It is 
easy to sec that s and r arc CSL- equivalent, but they are not strong bisimilar. ■ 

In Example 2 s ^ r would hold if one allows combining transitions with 
different exit rates, but unfortunately this does not work generally, refer to Ex- 
ample 3. 

Example 3. Suppose that we let s Ap /i iff there exists {s tJ,i\i(^i and {pi}iizj 
such that J2ieiPi ~ J2i£iPi ■ K — ^ and J2i£iPi ' Mi ^ M- Given two states s 

3 4 5 3 5 

and r such that s jii, s — > /i2; s — >■ /ia, and r — > /ii, r — > /i3 where /ii(si) = 
0.3, Aii(s2) = 0.7, /i2(si) = 0.4, Ai2(s2) = 0.6, M3(si) = 0.5, and /i3(s2) = 0.5. 
For simplicity again we assume that si, S2, and S3 are absorbing states and all 
the states have different atomic propositions except L{s) = L{r), then s and r 
should be bisimilar. But there exists a formula tp such that r \= tp and s ^ f. 
For instance let ip = L{si) with / = [a, 00), then the maximum probability of 
the paths starting from s satisfying is max{0.3 • e~^°, 0.4 • e^^", 0.5 • e^^°}. If 
G (|, |), then maximum probability is 0.4 • e~^° which is obviously greater 
than the maximum probability of the paths of r satisfying ip. M 

Below we show that the bisimulation relation is a congruence with respect 
to the parallel operator we introduced in Section 3: 

Theorem 2. s ^ r implies that s\\t ^ r\\t for any t. M 
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4.2 Weak Bisimulation 

In this section we will introduce a novel notion of weak bisimulation for CTMDPs 
in the sense that it only preserves CSL\^x equivalence. Our definition of weak 
bisimulation is directly motivated by the two examples in the previous section, 
together with the well-known fact that uniformization does not alter reachabili- 
ties for CTMDPs [6,23,22]. Even though we have seen that strong bisimulation 
is sound but not complete with respect to CSL equivalence, we can show that 
the two relations do agree on a subclass of uniformizcd CTMDPs. As a result, 
the weak bisimulation is both sound and complete for the sub- logic CSL\x for 
the same subclass of CTMDPs (not necessarily uniformed). The section ends up 
with a discussion about why the results do not hold for general CTMDPs, and 
motivates the study of a sequence of bisimulations in next section. 
Below follows the definition of weak bisimulation. 

Definition 6 (Weak bisimulation). We say that states s and r are weak 
bisimilar, denoted by s « r, whenever s ^ r in the uniformized CTMDP C. I 

Our weak bisimulation is a conservative extension of strong bisimulation. The 
following lemma establishes a few properties: 

Lemma 1. 1. ^ C 

2. for uniformized CTMDP, ^ = «. 

3. s ~cSL\x r in C iff s ~csl r inC. ■ 

Now wc shall show that, different from the strong bisimulation, « coincides 
with '^cSL^x ^ subclass of CTMDPs, which is defined in the following. 

Definition 7. Let TZ be an equivalence relation on S . A state s is said to be 
2-step recurrent w.r.t. TZ iff s is not absorbing, and moreover \Suc{s)\ > 2 and 

3(s ^ fi).{3{s' e Supp{^l)).iyis' ^ iy).i^{C) = 1)) 

where C = {^t&supp(n)[A'R)^[s\'R. and [s\ti = {r \ s TZ r} is the equivalence class 
of TZ containing s. We say C is 2-step recurrent w.r.t. TZ, iff there exists a state 
s G S which is 2-step recurrent w.r.t. TZ. I 

The non 2-step recurrent states can be seen as an extension of the well-known 
non-absorbing states, those that can evolve into other equivalence classes. Non 
2-step recurrent states extend non-absorbing states further by excluding those 
non-absorbing states that can evolve into other equivalence classes only through 
their parent and the parents' equivalent states. Moreover, we say that s (or C) 
is 2-step recurrent iff it is 2-step recurrent w.r.t. '^csl^x- Intuitively, the term 
2-step recurrent requires that s has more than 2 successors and there exists a 
transition s — > /x such that some states in Supp{pL) must only return back to 
states equivalent to s or states in Supp{iJ.) directly. We show below « coincides 
with CSL\x for CTMDPs without 2-step recurrent states. 

Theorem 3. ~ C ~csL\x- '■s "■ot 2-step recurrent, ~ — ~csL\x- ' 
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Fig. 2. Counter example of strong probabilistic bisimulation. 

In the proof we only need to use unbounded until, A (to construct the mas- 
ter formula of each equivalence class), and V. Thus, the following sub-logic is 
sufficient to characterize weak bisimulation for CTMDPs which are not 2-step 
recurrent: 

ip ::— a \ ip f\ ip \ ip \J ip \ V^p{il)) tp ::= ipU' ip 

Below we show that, as for strong bisimulations, the weak bisimulation re- 
lation is a congruence with respect to the parallel operator we introduced in 
Section 3. Moreover, for CTMDPs which are not 2-step recurrent, ^csL\x ^ 
congruence as well. 

Theorem 4. 1. s k, r implies that s\\t « r\\t for any t. 
2. if C is not 2-step recurrent, s ~cSL\x ^ implies that s\\t '^cSLyx 1 1 ^ /'^'^ 
any t. ■ 

General CTMDPs. The following example explains the necessity to consider 
CTMDPs without 2-step recurrent states in Theorem 3. It is shown that when 
2-step recurrent states are involved, ^csl^x — ~ does not always hold. 

Example 4- Suppose we are given two states sq and ro of a CTMDP depicted in 
Fig. 2. 

First assume s,; and are absorbing states for each 1 < i < 3. In this case, 
it is easy to check that sq and rg are 2-step recurrent states where all the states 
have different atomic propositions except L{si) = L{ri) for each < i < 3. ^ 
Then there does not exist a CSL formula which can distinguish them, as a result 
they are CSL equivalent. On the other hand, sq and rg arc not bisimilar, as for 
the middle transition of rp, sq has no way to simulate it even with combined 
transition. 

Now suppose that S2 and r2 are not absorbing, for instance they can evolve 
into So and with probability 1 respectively, then still they are CSL equivalent. 



^ Assume that TZ — {{s,r) \ L{s) = L{r)}, and same for the following examples. 
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But interestingly, if the non-absorbing states are S3 and instead but with the 
same transitions, then sq -^csl ''o- Considering the formula 

the maximum probability of the paths of sg satisfying V' is f while this proba- 
bility in rp is I > |, thus sq |= P<5-0 but rp ^ P<5-0. Note that even we let S2 
and S3 have such transition, sp and rp are still 2-step recurrent by Definition 7. 



The key idea behind the difference illustrated in Example 4 is that the bisim- 
ulation relation only takes one step into consideration. This restriction might be 
the best one can hope for the completeness results. 



4.3 Determining 2-step Recurrent CTMDPs 

In Theorem 3, the completeness holds only for CTMDPs which are not 2-step 
recurrent. This section discusses a simple procedure for checking it. The following 
lemma holds by applying the definition directly: 

Lemma 2. Given two equivalence relations TZ and TZ' over S such that TZ <^ TZ' , 
then if C is 2-step recurrent w.r.t. TZ, then it is 2-step recurrent w.r.t TZ' , or 
equivalently if C is not 2-step recurrent w.r.t TZ' , then it is not 2-step recurrent 
w.r.t. TZ. I 

Lemma 2 suggests a simple way to check whether a given CTMDP C is 2-step 
recurrent w.r.t. ^csLyx- know that ~ C ~ C '^csl^x — where 
72. = {(s,r) I L{s) — L{ry}. By Lemma 2, we can first check whether C is 2-step 
recurrent w.r.t. TZ, if it is not, we know that C is not 2-step recurrent either w.r.t. 
"^csL\x- Otherwise we continue to check whether C is 2-step recurrent w.r.t. ~ 
or if the answer is yes, then C is 2-step recurrent too w.r.t. '^csl^x- Both ~ 
and K, can be computed in polynomial time, see [32] for detail. 

In the remaining cases, namely when C is 2-step recurrent w.r.t. but 
not w.r.t. TZ, we cannot conclude anything, thus the relation ^csl^x shall be 
computed first for this purpose. The decision algorithm for ~csL\x falls, however, 
out of the scope of this paper. 



5 Characterization of CSL in General CTMDPs 

In [27] wc have defined a sequence of strong bisimulations to characterize prob- 
abilistic CTL (PCTL) as well as its sub-logics. Following that approach, in this 
section we show that such strong bisimulations can be used to characterize CSL 
and its sub-logics as well, for general CTMDPs. 
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5.1 Strong i-depth Bisimulations 

For the interval / = [a, 6], define I Q x = [a — x,b — x] if a > x, and I Q x = 
[0,6 — x] if a < .T < &. First, we define the notation ProbTr.s{C,C' ,n, I,uj), 
denoting the probability of reaching C", from state s, via only states in C 
within time in the interval / C [0, oo) and in at most n steps under sched- 
uler TT, where uj is used to keep track of the path having been visited. Formally, 
p = Prob-^,s{C, C", n, [a, 6], i^) is defined as follows: 

1. if (n = 0) A (a = 0) A (s G C"), P is equal to 1 

2. else if (s e C A s ^ C") A (n > 0), p is equal to 



4. otherwise p is equal to 0. 

The above definition has the same flavor as the definitions in [1, 20] - extended 
with bounds on the discrete steps. The first clause is trivial. For the second 
clause, s & C A s ^ C and we have still steps n > 0. The term n{uj,X,fi) 
denotes the probability that the pair (A,/x) is chosen by the scheduler tt under 
consideration. Further, A • e"'*'^ is the density of leaving s at time x. Once s is 
left, the successor s' is taken with probability /x(s'), from which we have n — 1 
steps and [a, b] Q x time left. The path is then augmented with the pair {x, s'). 
For the third clause with (s G C n C") A (n > 0), either we stay in state s more 
than a time units with probability J°° \ ■ e~^^dx = e~'^", otherwise we should 
continue, and the argument is the same as the previous case. For all the other 
cases, it is obvious that the result equals 0. Below follows the definition of strong 
i-depth bisimulation where s '^o r iS L(s) — L{r): 

Definition 8. A relation TZ S x S is a strong i-depth bisimulation with i > 
if s TZ r implies s ^i-i r and for any TZ downward closed sets C,C' and I, 

1. for each scheduler n, there exists a scheduler tt' such that 




J2 ^(c.,A,m)-(^ A-e-^-. ^(*') 



• Prob^ gi{C, C", n — 1, [a, 6] .t, a;^(x, s'))dx 



3. else if (s G C n C") A (n > 0), p is equal to 




Prob^,,r{C, C',i, /, r) < Prob^,s{C, C , i, /, s), 



2. for each scheduler tt, there exists a scheduler tt' such that 



Prob^>,s{C, C',iJ,s) < Prob^^riC,C\i,I,r). 
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We write s r whenever there is a strong i- depth bisimulation TZ such that 
sTZr. U 

It is not hard to show that is an equivalence relation. 

Lemma 3. '^i is an equivalence relation for all i > 0. M 

Similarly we can show that is both sound and complete for ^^sl^ i ^^.d 
also in an arbitrary CTMDP there exists a n such that = ~csl, therefore 
we have the following theorem. 

Theorem 5. 1. = '^csl" ■ 

2. There exists n such that ~ '^csl- ^ '^csl- 

3. '^i with i > 1 is not in general a congruence (w.r.t the operator \\ in Defini- 
tion 4)- B 

The following example illustrates that is both sound and complete for 
'^CSL" even for general CTMDPs. 

Example 5. Refcr to sq and ro in Example 4. If Sj and rj are absorbing states 
with 1 < J < 3, then it can be proved that sq rg, thus sq '^csl" ''o 

for any i > 0. Similarly for the case when S2 and r2 are not absorbing but 
can evolve into Sq and ro with probability 1 respectively. Suppose now the 
non-absorbing states are S3 and r^ with same transitions, then we show that 
there exists n such that sq oo^^ rg. Let C ~ {sq, s^^r^^r^} , C = {si,ri}, 
and / = [0,cxd), then it is easy to see that ProbTri,so{C,C' ,2n + 1,/, sq) = 

0. 3 • X]r=o^-'^* where tt; always chooses the left transition when at so- Simi- 
larly Pro6,r^,so(C, C", 2n + 1,/, so) = 0.5 • X^ILo*^-!* where tt^ always chooses 
the right transition when at so. Given a iTm which always chooses the middle 
transition of ro when at rg, then Pro^Tr^.m (C, C ,2n+l,I, rg) = 0.4 • X^ILo 
Observe that hm„^oo ^'ro&7r,„,ro (C: C", 2n -I- 1,/, ro) = | which is greater than 
lim„^oo ^'ro&7r,,so(C, C",2n -I- 1,/, So) = 5 and lim„^oo -Pro&^^^s,, (C, C", 2rt -|- 

1, /, So) = |, thus there must exists n such that it holds Prob-j^^ ra {C, C' , n, I,rQ) > 
Proh^i^soiC, C", n, /, so) and Prob^^^ro (C, C", n, /, ro) > Prob^^^sa (C> C' , n, I, so), 

thus So '^n ■ 

Recall that CSL\u„ denotes the sub-logic of CSL without bounded until. The 
following lemma shows that the intersection of ~ and ~i is sound and complete 
for this sub-logic: 

Lemma 4. // C is not 2-step recurrent, we have 

2. -^csL^u '^'-'^ general a congruence. I 

The example below shows that Lemma 4 does not hold in CTMDPs with 2-step 
recurrent states: 

Example 6. Again considering sq and rg in Example 4, if si and r^ are absorbing 
states for 1 < * < 3, then both so and ro arc 2-step recurrent states by Defi- 
nition 7. As we said before so -^csl ^"0, thus so ~csl\u„ but sq 9^ r^. 
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5.2 Weak i-depth Bisimulation 

Following the idea of defining weak bisimulations in Section 4.2, in this section, 
we introduce weak i-depth bisimulations. 

Definition 9. We say that states s and r are weak i-depth bisimilar, denoted 
by s ~i r, whenever s ~i f in the uniformized CTMDP C. I 

Due to that CSL\x satisfaction is preserved after uniformization, we have the 
following characterization results for CSL^x in arbitrary CTMDP. 

Theorem 6. 1. There exists n such that = "^csL\x- 
2. ~i is congruent, and ~i with i > I is not in general a congruence. I 

We have seen that ^csl\x ^ congruence in CTMDPs that arc not 2-step 
recurrent. Since ~i with i > 1 are not congruent in general, it follows that ^csl^x 
is also not congruent in general. 

6 Simulations 

In this section we introduce (weak) simulations, and i-depth (weak) simulations. 
Further, we extend the characterization results to these simulation relations. 

6.1 Strong and Weak Simulations 

To introduce the definition of simulations, we make use of the notion of weight 
functions in the way as [18]. 

Definition 10 (Weight function). LetTZ be a relation over S. A weight func- 
tion for fi and v with respect to TZ is a function Z\ : 5* x S* t— >■ [0, 1] such that: 

— A{s, r) > implies that s TZ r, 

- v{r) = Y^ses "^(^' ^""^ °''^y T e S. 

We write /i v iff there exists a weight function for fi and v with respect to 

n. u 

Now we extend the strong (weak) bisimulations to strong (weak) simulations 
for CTMDPs, respectively: 

Definition 11 (Simulations). Let TZCSxS,TZisa strong simulation if 
s TZ r implies that for each s ^ ^, there exists r ^' such that /i fi' . 

We write s < r whenever there exists a strong simulation TZ such that s TZ r. 

We say that s is weak simulated by r, denoted by s ^ r, whenever s ^ f 
in the uniformized CTMDP C. ■ 
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The relation ~< is then a preorder. To characterize -<, we use the safe fragment 
of CSL [3], denoted as CSL^, which is defined by the following BNFs: 

ip -.-.—a \ \ (p A (fi \ (py (fi \ V>p{4') 
iP ::=XV I V3U^^ I ip\}^ ip 

As usual, CSLjyx is obtained from CSLs by removing the next operator. Below 
we present the logical characterization results for strong and weak simulations 
with respect to CSL^ and CSL^^Xj ^-nd their relationship: 

Theorem 7. 1. < C ^csl, • 

2. if C is uniformized and not 2-step recurrent, ^cSL, = 

3. ^ C ^CSL,,,. 

4-. if C is not 2-step recurrent, -<cSL3\x ~ ' 

Example 4 can applies here as well showing that Theorem 7 does not hold in 
general CTMDPs. Let denote the reverse of the relation R. The following 
theorem shows the compositional properties and their relation to bisimulations: 

Theorem 8. 1. s ^ r implies that s\\t ~< r\\t for any t. 

2. s ^ r implies that s\\t ^ r\\t for any t. 

3. If C is uniformized, -< = and ^cSL^ = -^CSL^^x- 

4- If C is not 2-step recurrent, s ^csl^^x implies that s\\t ^csl^^x ^11^ 
any t. 

5. - c (-^ n 

6. ^ c n ^ -1). ■ 



6.2 Strong and Weak i-depth Simulations 

In this section we introduce the one side strong and weak i-depth bisimulations. 
Below follows their definitions where s -<o r iff L{s) = L(r): 

Definition 12 (t-depth simulations). A relation TZ C S x S is a strong i- 
depth simulation with i > if s TZ r implies s -<i_i r and for any TZ downward 
closed sets C,C',I and n, there exists a scheduler n' such that 

Prob^,^r{C,C',t,I,r) < Prob^^s{C,C',i,I, s). 

We write s <i r whenever there is a strong i-depth simulation IZ such that 
s TZ r. 

We say that s is weak simulated by r, denoted by s r, whenever s ~<i f 
in the uniformed CTMDP C. I 

The following theorem shows the properties of -<i and especially there 
exists n such that -<„ and ^„ are enough to characterize CSLs and CSL^yx 
respectively. 

Theorem 9. 1. ~<i is preorder, and ~<i ~ ^qqa - ■ 
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2. There exists n such that -<n — "<cSL- = ^cSLs, and — ^cSL.,\x 
in any CTMDP. 

3. -<i with i > 1 and with i > 1 are not congruences while is a congru- 
ence. ■ 

As a direct consequence, -<cs\-^-^s^ ~ ~i is not congruent for i > 1. Below 
we prove a few properties of i-depth simulations, along Theorem 8: 

Theorem 10. 1. If C is uniformized, <i = 

3. c (^.n^ri). ■ 

Extending Lemma 4 to simulations, we can also characterize CSL^^u^ i.e. the 
safe CSL without bounded until. 



Lemma 5. If C is not 2-step recurrent, we have 
~ n -<i = ^CSL,^u„. 

2. ^csL^vu ''T'Ot a congruence. 



7 Relation to Probabilistic Automata and Markov chains 



In this section we discuss the relation of our bisimulations with those in the 
embedded time-abstract models. 



7.1 Relation to Bisimulation of Probabilistic Automata 

Let C be a CTMDP, the embedded probabilistic automata Aic is obtained by 
removing the rates on the transition relations. In probabilistic automata (PA), 
probabilistic bisimulation ^p, and branching bisimulations (up to i) are defined 
[25, 27], and are denoted by '^p, ^\ respectively - they are recalled in Appendix 
D.3 in the appendix for the readers convenience. The following lemma is obvious 
from the definitions: 



Lemma 6. 1. s ^ r implies s ~p r in Mq. 

2. If C is uniformized, then s ~p r in A4c implies s ^ r. 

3. s r implies s ^\ r in A4c- 

4. If C is uniformized, then s r in A4c implies s r. 



The other direction for the first clause does not hold generally. For PAs, we 
know that ~p is only sound but not complete for PCTL, so it is a surprise that 
the strong probabilistic bisimulation in the continuous setting with minor variant 
is both sound and complete for CSL in the uniformized CTMDPs without 2-step 
recurrent states according to Definition 6 and Theorem 3. Refer to Example 7 
for an intuitive explanation. 



16 Lei Song, Lijun Zhang, and Jens Chr. Godskesen 



Example 7. Considering two states sq and of a PA in Fig. 2. Suppose that Si 
and ri can evolve into t with probabihty 1 where 1 < « < 3 and t is absorbing. 
Also all the states have different atomic propositions except L{si) ~ L{ri) for 
< i < 3. It is easy to check that sq and are PCTL-equivalent, but sq '^p 
since the middle transition of rg has no way to be simulated by any (combined) 
transition of sq. Assume that so and as two states of a CTMDP where each 
transition has rate 1, then obviously the CTMDP is not 2-step recurrent by 
Definition 7. We can show that actually so and ro are not CSL-equivalent. Let 

V' = (i(so)VL(si))Ul'''''l(i(s3)VL(t)) 

where a state is used as a shorthand of the atomic propositions it satisfies. If sq 
chooses the transition on the left first, then the probability of the paths satisfying 
tp is equal to 

0.4 • (e"'' - e-'') + 0.3 • (a • e"'' + e"'^ - & • e"'' - e"''). 

The probability for other transitions can be obtained in a similar way by substi- 
tuting 0.3 and 0.4 with corresponding probabilities. Since the interval [a, b] can 
be chosen arbitrarily, so we can choose the intervals such that the probability 
of path satisfying ij: when choosing the middle transition of r is larger than the 
other two eases. For instance here we can choose interval oo), then the max- 
imum probability of paths of tq satisfying -0 is 0.9 • while the corresponding 
maximum probability of sq is only 0.85 • e^2^ so essentially sq and ro are not 
CSL-equivalent. ■ 

Different from the discrete case where ^\ is congruent, in the continuous case 
even ~i is not congruent, refer to the following example. 

Example 8. Considering s and r in Example 2, s and r are CSL-equivalent, thus 
s ~i r. Suppose we have t such that t can only evolve into ti with rate 2. 
We can show that actually s\\t oo-^ r\\t where all the states have different 
atomic propositions except L{s) = L(r). Let -0 = L{s) U((L(si \\t)) with / = 
[a, oo), then the probability of the paths of s\\t satisfying tj: by choosing the 
left transition is equal to | • e^^°, similarly the probability is equal to | • e^^° 
and I • e~^° by choosing the middle and left transition respectively. By solving 
the inequations: | • e~^° > | ■ e~^° and | • e~^° > | ■ e""^", we can see that if 
e~" e (y|, '^), the probability by choosing the middle transition is maximum 
which is greater than the correspondent probability of r, thus s\\t r\\t, 
and ^1 is not congruent. ■ 

7.2 Relation to Simulation of Probabilistic Automata 

Let and denote the strong probabilistic simulation and strong i-depth 
branching simulation introduced in [25] and [27, 28] respectively which are also 
recalled in Appendix D.3. The following lemma states the relation between ^p, 
and their continuous counterparts, which is similar to the bisimulation cases. 
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Lemma 7. 1. s ^ r implies s ~<p r in Mc- 

2. If C is uniformized, then s ~<p r in A4c implies s ~< r. 

3. s <i r implies s r in Mc- 

4- If C is uniformized, then s -<\ r in Aic implies s <i r. 



7.3 Relation to (Weak) Bisimulation for CTMCs 

For CTMCs each state has a unique Markovian transition, which will be denoted 

by s jjLs- The notion of weak bisimulation can be found in [3] for CTMCs, 
repeated as follows: 

Definition 13. For CTMCs, an equivalence relation TZ is a weak bisimulation 
iff for all s TZ r it holds (i) L{s) = L{r), and (ii) \s ■ tJ-s{C) — Xr ■ fJ-r{C) for all 
equivalence classes C ^ [s\ti. 

States s,r are weak bisimilar, denoted by s ~ctmc r, iff there exists a weak 
bisimulation TZ such that s TZ r. M 

Strong bisimilarity for CTMCs is defined if in addition As • ^s{C) ~ Xr ■ fJ-riC) 
holds for C = [s]ti = [rj-ji as well. States s,r are strong bisimilar, denoted by 
s ~CTMC iff there exists a strong bisimulation TZ such that s TZ r. 

Below we prove that, restricted to CTMCs, our strong and weak bisimulations 
agree with the strong and weak bisimulations for CTMCs: 

Lemma 8. For CTMCs, it holds that ^ = '^ctmc o-nd ?» = ~ctmc- ^ 

The lemma above shows that ^ and w are conservative extensions of the 
strong bisimulation and the weak bisimulation for CTMCs in [3], and so are 
their logical characterization results except that they only work in a subset of 
CTMDPs. 



7.4 Relation to (Weak) Simulations for CTMCs 

The strong and weak simulations were introduced in [3] , we recall the definition 
of the strong simulation as follows. 

Definition 14. For CTMCs , a relation TZ is a strong simulation iff for all s TZ r 
it holds (i) L[s) = L{r), (ii) fig C^j. /x^, and (Hi) Xg < X^. 

State s is strongly simulated by r, denoted by s -<ctmc iff there exists a 
strong simulation TZ such that s TZ r. ■ 

The following relation holds for simulations: 

Lemma 9. For CTMCs, -< C -<ctmc- If the CTMC is uniformized, 
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t J it 
Fig. 3. ^CTMC is too coarse (transition of t is omitted). 



The simulation relation ^ctmc in [2] is strictly coarser than ours. In [2], it is 
shown that -<ctmc characterizes a sublogic of ^csl^ , denoted by -<csl''j i^ which 
all intervals are of the form [0, b], i.e., the left endpoint is always 0. The following 
example illustrates this difference: 

Example 9. Considering the states s, r and t in Fig. 3 where L{s) = L{r) ^ L{t)^ 
and t is an absorbing state. According to Definition 14, it is easy to check that 
s -(cTMC r, but s T^cSLs r. Let -0 = {L{s) u'"'''' L{t)), then the probability 
for the paths of s and r satisfying -0 is equal to (e^^" — e^^*") and (e^"*" — e^^'') 
respectively, when a = and b > 0, {1 — e~'^^) < (1 — e^""'), while when a > 
and b = oo, e^^" > e^^". In the other word, there exists (f and Lp' such that 
s \= Lp, r Y= Lp and s ^ Lp' , r \^ ip' . Essentially, neither s -<csLs nor r -<csLs s 
holds. ° ° ■ 

The various strong simulation definitions in this paper can be slightly adapted 
such that they correspond to the safe sublogic as in [2]. However, the same does 
not hold for weak simulations, which is introduced in [2] for CTMCs - which is 
denoted by ^ctmc below. The relation ^ctmc is shown to be sound with respect 
to the sublogic CSL^^x (obtained from CSL° by removing the next operator). 
The completeness was conjectured, but remains open. In the following example 
we show that, on the contrary, the completeness does not hold. 

Example 10. Consider the two states sq and ro in Fig. 10, where different colors 
are used to indicate different atomic propositions. Assume that all the states 
only have transition to themselves except sq and rg. It holds that sq ^ctmc tq: 
briefly, ri is marked with red thus can not be matched by sq and its derivations, 
this is however required as both sq and tq perform visible (to states with different 
labels) steps. On the other hand, we observe that Sq ^csl" ''o- the transition 
to vi is silent, and to ui can always be simulated. Thus ^ctmc is not complete 

W.r.t. ^rc:i ■ 

For interested readers, detailed discussions are given in Appendix D.4, where 
we recall the weak simulation relation in [2] and elaborate the example in detail. 



8 Related Work 

Logical characterizations of bisimulation have been studied extensively for stochas- 
tic models. For CTMCs the logic CSL characterizes bisimulations, while CSL 
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Fig. 4. A counterexample for the completeness of ^ctmc- 



without next-state formulas characterizes weak bisimulations [3] . Our results in 
this paper is a conservative extension for both strong and weak bisimulations. 
In [12], the results are extended to CTMCs with continuous state spaces. 

For CTMDPs, the first logical characterization result is presented in [20]. It is 
shown that strong bisimulation is sound, but not complete with respect to CSL 
equivalence. For the non-completeness please refer to Example 2 in this paper. 
In this paper, we introduced the weak bisimulation relation for CTMDPs. For a 
subclass of CTMDPs, i.e. without 2-step recurrent states, we have shown that 
the weak bisimulation is also complete for CSL\x-equivalence. 

For probabilistic automata PA, Hennessy-Milner logic has been extended to 
characterize bisimulations in [19,8,17]. In [11], Dcsharnais et al. have shown 
that weak bisimulation agrees with PCTL* equivalence for PAs. The most re- 
lated paper for PAs is our previous paper in [27], in which we have introduced 
bisimulations and z-depth bisimulations for characterizing logical equivalence 
induced by PCTL and sub-logics. This leads to the study of the i-depth bisim- 
ulation relations for CTMDPs in this paper. For uniformized CTMDPs, we have 
shown that they agree with the equivalences in the discrete setting. 



9 Summary 



The spectrum of the branching time relations and the logic equivalences are 
summarized in Fig. 5. The arrow should be interpreted as "imply". The labels 
U and ^ denote that the implication is only valid in a uniformized CTMDP, and 
a CTMDP without 2-step recurrent states respectively. We write C directly for 
r^C for readability where >C is a sub- logic of CSL. The index n appearing on the 
right plane is chosen according to Theorem 5 and 9. Thus ~ '^n for all 
k > n, and similar holds for other relations, and for a smaller index, the relation 
will be coarser. 

As future work we would like to consider the approximation of bisimulation 
and simulation on CTMDPs as well as their logic characterization, along [10]. 
Another interesting direction is to define bisimulation and simulation relations 
between distributions [7]. 




CSU " >■ csu 



Fig. 5. Relationship of various bisimulation and simulation relations 
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A Proofs of Section 4 
A.l Proof of Theorem 2 

Proof. Let TZ ^ {{s \ \t,r \ \t) \ s ^ r}, it is enough to show that 7?. is a strong 
bisimulation. Suppose that (s || TZ (r || t), and s\\t ^ fi. By Definition 4 there 
exists s and t v such that A = Ai + A2, /i = ^ • {^i \ \ Vt) + ^ • 

{T>s\\v)- Since s ^ r, there exists r — 4>p ji'^ such that ni ^ /i'^, thus 
(/ii II 2?t) (m'i II 2^*) I well as iVs II ''^ (A- \ \^)- As a result there exists 
r II t A>p fi' = ^ ■ (p'l II + 4f ' (■^'" 1 1 i^) , so /i 7?, /i' which completes the proof. 

A. 2 Proof of Lemma 1 

Proof. To show that ~ implies «. we observe that for each .s \ fi, wc have 

s (-^^ ■ + -5 ■ m)i similar for s /i. The following proof is trivial. 

The proof of Clause 2 is straightforward from Definition 6. 

We first prove that if C is a CTMC, then s ~^csL\x in C iff s -^csl f in C. 
Since uniformization preserves the satisfiability of CSL^x, thus s ^cs\-\^ ^- Let 
TZ = {(s, f) I s ^csLyx fi'^st prove that 7^ is a strong bisimulation. Let A 

denote the exit rate of s and f, and Aj denote the rate from s to states in [s]ti 

i.e. s ^ fj, and Aj = A • ^{[s]ti)- The case when Aj = A is trivial, we assume that 
A > As, then for each C € S/TZ such that s ^ C, the probability of the path of s 
satisfying ips u'"'*'! cpc is equal to ■ (e-^^ ° - e^^^ '') where Ac = A • n{C). 

Since s ^csl^x ^' must be the case such that f ^ ly with /i(C) = '^(C') i.e. 
^ TZ v, thus 7?. is a strong bisimulation. According to [3], s ~^csl f^- 

We now generalize the result to CTMDP. If s ^csl^x '^i then s '^csLi^x ^• 
Since in a uniformized CTMDP, every execution of C guided by a given scheduler 
can be seen as a CTMC, thus s ^csl ^ based on the above result. 

A.3 Proof of Theorem 3 

Proof. In the following the parameter E will be omitted in the transition, i.e. 
we simply write s ^ fi for s ji. 

First we show that « C ~csL\x- Let C be a CTMDP and assume s w r. 
By the definition of weak bisimulation, we have s w r in C. By Thm 1, s ^csl r 
in C. Applying the third claus of Lemma 1, it holds that s ^csl^x ^■ 

Now we prove that '^csl^x implies « whenever C is not 2-step recurrent. 
By definition, it is the same to prove that ~csl implies ^ in a uniformized 
CTMDP. In the following we assume that the given CTMDP is uniformized and 
assume that the rate is equal to 1 for simplicity without losing generality. Let 
7Z = {is,r) I s ^csL f} which is obviously an equivalence relation, we are 
going to show that 7?. is a strong bisimulation. By contradiction we assume that 
TZ is not a strong bisimulation, then there exists (s,r) G TZ such that either i) 
L{s) ^ L{r), or ii) there exists a s — )• /x such that there does not exist r — )>p v 
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with fiTZ i>. In both cases, if we can find a formula ip such that s \== ip but r ^ ip 
or the other way around, then we can obtain a contradiction. Case i) is easy and 
we only focus on ii) here. Suppose there exists a transition s ^ /i, since C is not 
2-step recurrent, there are three different cases to consider. 

1. s_L i.e. s is an absorbing state. This case is trivial since all the derivations of 
s will stay in the same equivalence class [s]fi. 

2. Suc{s) < 2 i.e. there exists at most two equivalence ^classes Ci,C2 G S/TZ 
such that fi{Ci U C2) = 1, in the other words, n{Ci) = 1 — fJ.{C2)- The 
reason to consider this special case is that for each /it, if there exists fii and 
fi2 such that fii{Ci) < fJ.{Ci) < /X2(Ci), then we can make sure that there 
exists wi, W2 such that wi + W2 = 1 and wi ■ /xi(Ci) + W2 • /^2(Ci) = l^-iCi), 
therefore 

Wl ■ fJ-l{C2) + W2 ■ fl2{C2) 
= Wi-{l- fli{Ci)) +W2-{1- MCl)) 
= Wi+W2- {Wi ■ Atl(Ci) + W2 ■ M2(Cl)) 
= l^fl{Ci)^fl{C2) 

thus {wi ■ fii + W2 ■ fJ-2) = fJ. as we expect. This cannot be generalized to the 
case where Suc{s) > 2. 

Let (fie be the master formula of an equivalence class C G S/TZ such that 
Sat{ipc) = C. Since s ^csl and s j= P>i(X'°'°°''(iy9c'j V ^02)) obviously, 
thus r h P>i(X["'°°^(^Ci V Lpc^)), that is, Suc(r) C Ci U C2 which means r 
can only move to states in Ci U C2 too. Secondly, we prove that there exists 
r ^ ui and r — > 1/2 such that vi{Ci) < /x(Ci) < 1/2 (Ci). Assume there does 
not exists r — > 1^2 such that 1^2(^1) > //(Ci), in the other words, for all r — > 
we have i'(Ci) < ^(Ci), so there exists q such that ?- |= P<g(X'"'°°-' (yScJ, but 
s P<(j(X[°'°°^ (pcj ) which contradicts with the assumption that s '^csl r. 
Similarly, we can show that there exists r — >• z^i such that 1^1 (Ci) < /i(Ci). 
Based on the discussion above, we can guarantee that there always exists wi 
and W2 such that wi + W2 ~ 1 and [wi ■ vi + W2 ■ V2) TZ fi. 

3. We consider the - most involved - remaining case: Suc{s) > 2 and for all 
s' £ Supp{fi), there exists t and s' — > fi' such that fi'{t) > where t is in a 
different equivalence class from which s and the states in Supp{fi) belong to. 
We prove by contraction. Assume that there does not exists r — >-p 1/ such 
that fi TZ v. Note every combined transition of r can be seen as a combined 
transition of two other combined transitions of r. We fix two arbitrarily fixed 
(combined) transitions of r: r — j-p vi and r — >p 1/2, thus 

VO < U)i, U'2 < l.Wi + U)2 = 1 (1) 

A ^ /!Z (wi ■ ui + W2 ■ V2) 



^ The Sues — 1 can be covered by taking C2 = in the proof. 
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Let Supp{fi) — {si, S2, ■ • ■ 7 Sn}- For simplicity wc assume that si, . . . , s„ 
belong to different equivalence classes. For 1 < i < n, define: 

^J.{s^) = a.i,i^i(sj) = 6j, and 1^2(81) = Q . 

Then there must exist 1 < fc 7^ j < n such that there docs not exist < 
Wi,W2 < 1 withu'i+ui2 = 1 such that Wi-6fc+W2-Cfe — aiidwi-bj+W2-Cj = 
aj, otherwise ^ TZ (lOiJ^i + 'W2V2) which contradicts Eq. 1. The idea now is 
then to construct a formula if which is satisfied by s but not r, depending 
on the relation between Ofe, Oj and 6fc, Cfc, hj.Cj. There are nine possible cases 
in total depending on whether au G [f'fe,Cfc] and/or aj G [6j,Cj]. Most of the 
cases are trivial except when au G Ck] and aj € [cj, bj] with Ck > bk and 
bj > Cj. ^ The formula for this case is given by: 

(p = {ry sV Sk)U^'''''\sjV s'^,) 

where sj. is the successor of Sk not equivalent to s and the states in Supp(fi), 
and the names of states are used as the abbreviations of the state formulas 
characterizing the equivalence classes where they are located. Suppose there 
exists Sk — fJ-k with = p, and define: 

pi^p-ia- e-" + e-" - b ■ e'^ - e"'') 

P2 = e-" - e-^ 

then 

— the probability of s satisfying tp by choosing transitions s — > yU and 
Sk — > Pk is equal to p(s, p) := aj ■ p2 + ak ■ pi, and 

— the probability of r satisfying <p by choosing the combined transition of 
r — > vi and r — >■ i^i and Sk Pk is either p{r, vi) :~ bj ■ p2 + 6fc • pi or 

P{r,V2) = Cj ■ P2+Ck- pi- 

Now it is sufficient to prove that we can find < a < 6 such that p{s, p) > 
p(r, I'l) and p{s,p) > p(r, 2/2). We claim that it is the case once we can 
guarantee ^ G ( . ), which can be seen as follows: 

— Let 6 = 00, then ^ = p • (a + 1) and it is easy to see that there exists 
a, b such that ^ G [p, 00). 

— On the other hand let a = 0, then pi = p ■ {1 — e^'' — b ■ e^^) and 
P2 = 1 - so ^ = p ■ (1 - ^i^), note here that e (0, 1) since 
^'^^-h can be arbitrary close to 1 when b is close to 0, and in the other 
hand, i_^i,-b is arbitrary close to as 6 increases. As a result ^ G (0, p). 

® For instance if a;, > bk,Ck, s will evolve into Sk with higher probability than r, so 
is easy to give. 

^ By solving two equations:afe ■ pi + aj ■ p2 > bk ■ pi + bj ■ p2, and • pi + Oj ■ p2 > 
Ck ■ pi+ Cj ■ p2, such pi and p2 always exists due to that there does not exists W\,W2 
such that wi ■ hk + W2 ■ Ck = ak and w\ ■ bj + W2 ■ Cj = aj. 
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— it is not possible for — — = — — —, otherwise there exists < wi, < 1 
such that wi-h}^+W2- Ck — o-k and wi -bj + W2- Cj ~ Uj with W1+W2 — 1 • 
Thus there always exists < a < 6 such that s will satisfy ip with higher 
probability than r for some a, b, therefore s '^''CSL ^, and we have a contra- 
diction. All the other cases are similar and omitted here. 

A. 4 Proof of Theorem 4 

Proof. We prove Clause 1 first. Since s w r, by Definition 6 s ~ f 
to Theorem 2 s 1 1 f ~ f 1 1 ? for any t. It is easy to check that s 1 1 i = 
result s\\t ~ r\\t which implies that s\\t ?» r\\t for any t. 
The proof of Clause 2 is straightforward based on Clause 1. 

A. 5 Proof of Lemma 2 

Proof. Straightforward from Definition 7 and the fact the [s]n C [sj^/ provided 

that n C n'. 

B Proofs of Section 5 

B. l Proof of Lemma 3 

Proof. The reflexivity and symmetry are easy to show, we only prove the transi- 
tivity here. Suppose that s t and t ~i r, we should prove that s ~i r. By 
Definition 8 there exists two strong i-depth bisimulation TZi and 1^2 such that 
s TZi t and 1 1^2 r. Let Tl^TZ\o7l2 = {(si, S3) | 3s2-(si T^i S2 A S2 7^2 r)}, it is 
enough to show that 7?. is a strong i-depth bisimulation. Note TZi U7^2 ^ TZ, since 
for each si TZi S2 we also have S2 7^2 S2 due to reflexivity, thus Si 7?. S2, similarly 
we can show that TZ2 C TZ. Therefore for any TZ downward closed sets C and 
C", they are also TZi and TZ2 downward closed. As a result for each / and tt, there 
exists tt' such that Prob.,^: ^t{C,C' I ,t) < ProbTr^s{C,C' ,i,I,s) since s t. 
Furthermore, since t r there exists tt" such that ProbT^n ^r{C,C' ,i, I,r) < 
ProbTri^t{C,C' ,i, I,t). This completes the proof. 

B.2 Proof of Theorem 5 

Proof. Let Sat{ip) = {s G 5 | s |= </?} denote the set of states satisfying ip, 
and Sat{tp) = G Paths'^ \ uj \= ^} denote the set of paths satisfying ip. We 
prove that ^^sl- ^ first. Let TZ = {{s^r) \ s ^csl^ is enough 

to show that 7?. is a strong i-depth bisimulation. It is a standard technique to 
construct a state formula Lpc such that Sat{ipc) = C where C is 7?. downward 
closed. Suppose that there exists tt, C, C" and / such that there does not exist 
a scheduler tt' with Probj^' ^r{C,C' ,i, I,r) < Prob.,^^s{C,C' ,i,I,s) where C, C" 
are TZ downward closed sets, and / C [0, cg) is an interval. In the other word 



. According 
— s 1 1 as a 
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ProbT,>^riC,C',i,I,r) > Prob„,s{C,C' ,i,I,s) for any tt'. Let /'r^,s((^i U[ 1^2) de- 
note the probability of the paths of s satisfying ipi U[ ip2 guarded by the sched- 
uler TT, it is not hard to see that PrT^,s{ipc ^pc) = ProbT^^siC.C ,i,I,s). As 
a result there exists q such that r ^ V>q{Lpc U,; ^pc) but s ^ V>q{ipc Uj (/3c') 
which contradicts with our assumption, therefore there does exist tt' such that 
Prob^>,r{C,C\i,I,r) < Prob^^siC,C' ,i, I , s), thus s TZ r. 

In order to prove that C ^csl" ' need to show that for all states 
s and r, s \^ if implies r \= if and vice versa whenever s r, where if 

is any state formula of CSL~. We only consider formula P<q{il)) here since all 
the others are trivial. Suppose ip = ip where / — [a, 6]. We show that the 
next operator can be encoded by bounded until. First consider the case when 
s ^ Sat{ip), then Pr^ siYJ ip) — Pr.,^^s{^s U( lys) for any scheduler tt. Suppose that 
s G 5ai(^). Since Pr.,.(XV) - E(A,p)6Supp(.(.)) A^) " (e"^"^ " e"^") - 

PrTr,sO^^ ^v)^ SO we can use the above result to encode Pr^ ,j(X^ -Kys) as well. 
As a result we only need to consider the case when ip = (piUlip2- Suppose 
that s \= P>q{ip), that is, V7r.Pr^_s(v3i U[ 1^2) > q- Since Fr^^s((pi U,[ 932) = 
ProbT^ ^s{Sat{ipi) ^ Sat{Lp2), i, I, s) for any scheduler tt, then we have 

\/'K.ProbT,^s{Sat{ipi), Sat{ip2), i, I, s) > q. 

Again we prove by contradiction, assume that r ^ P>qip, then there exists tt' 
such that ProbT^>_r{Sat{ipi), Sat{ip2),i, I ,r) < q, since s r, then there should 
exist TT such that 

ProbT^,s{Sat{ipi), Sat{ip2),i, I, s) < ProbT^> ,r{Sat{ipi), Sat{ip2),i, 1 ,1") < q, 

this contradicts with the fact that s \= P>qip, so r |= P>qip, this completes our 
proof. 

The proof of Clause 2 is trivial since there are at most n equivalence classes 
where n is the number of states in a CTMDP, thus ^„ ~ "^csl- = '^csl- 
For the counterexample of the last clause please refer to Example 8. 

B.3 Proof of Lemma 4 

Proof. By Theorem 3 ~ = '-^csl^x ^ ^"'"'^'-"^ ^"S^^P s^^^^^' 

and moreover by Theorem 5 ^1 — ^csl^ ■ ^^'-0 denote the sub-logic of 
CSL without (bounded and unbounded) until operator. We are going to show 
that CSLq" = CSL]^. The proof of ^^sl^ ^ '^csl" trivial. We show that 
~f-5i_- C -^csL- ■ The only case we need to consider is tp = P<q{ipi U( ip2)- We 
prove by structural induction on (p. Suppose that s \= (p and s \= (pi A-np2, if we 

choose transition s A- /x, then the probability of the paths of s satisfying (pi U{ ip2 
is equal to fj,{Sat{ip2)) ■ {e~^°' — e"'*''') where / = [a,b], note the probability of 
the paths of s satisfying ip2 is also equal to ^{Sat{ip2)) ■ (e~'^" — e"'*'''), in the 
other words, if s \= ipi A ~'(p2, then s \= ip iS s \= P<q{X^ (^2)- Since s ~(;sl~ 
then r ^ P<qO^^ V2), by induction r \^ (pi A -^ip2, thus r \= ip. The other cases 
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arc similar and omitted here. Therefore « n ~i is both sound and complete for 
Since -^i is not congruent, the first clause implies clause 2 directly. 

B. 4 Proof of Theorem 6 

Proof. The proof of the first clause is based on Theorem 5. We first shows that 
s ~ 7' implies that s ~csL\x i-C ~ ^ ^csl>^x- ^i^*^*^ * ~ then s ^ f , thus 
s ~csL\x ^ by Theorem 5. Since uniformization does not change the satisfaction 
of CSLyx, therefore s '^csl^x ^- show that '^csl^x — ~' prove that 
s '^cSL^x implies that s w r. It is easy to sec that ~csL\x = '^csl in a 
uniformized CTMDP, thus s '^cSLyx implies that s ^csi ^- By Theorem 5 
s ~ f, therefore s « r. 

We prove that Wi is congruent. By Definition 9, s ~i r iff s ~i f, so we 
only need to show that is congruent in uniformized CTMDPs. It is enough 
to show that TZ = {(s||t,r||t) \ s ~i r} is a strong 1-step bisimulation. 
Note that in a uniformized CTMDP s r iff for each ^i-closed set and 

s — >■ /X, there exists r v such that i^(C) < /i(C) and vice versa. Suppose 
that s\\t —> ^, by Definition 4, there exists s ^ iig and t ^ iit such that 
^ = i • (^^ II Vt) + ^ • (2?i5 II /it), the following proof is straightforward. 

C Proofs of Section 6 

C. l Proof of Theorem 7 

Proof. In order to show that -<csLs Q -< when C is not 2-step recurrent, 
it is sufficient to show that TZ = {{s,r) \ s -<csLs t} is a strong simulation. 
Suppose that s TZ r and s — >-p fi, we need to show that there exists r — >p v 
such that fi V. Similar with the proof of Theorem 3, if there does not exist 
r — ^-p V such that fi ^-r v, then a path formula "0 and tt can be found such that 
Prr^TT'i'tp) > -Prs,7r('0) for all tt'. Therefore there exists q such that r ^ P>q'4' but 
s P>qil), which contradicts our assumption that s ^csl^ t- 

Now suppose that s ~< r, we are going to show that s ^csl^ f, that is, 
r \= implies s \= ip for any ip of CSL^ by structural induction on Lp. First we 
show for each tt of s, two -< downward closed sets C,C', and / = [a,b], there 
exists tt' of r such that ProbTr'^r{C,C' , I,r) < ProbTr^s{C,C' , I, s). Since C and 
C are -< downward closed, there exists (pc and (pc such that Sat(ipc) = C and 
Sat{(pc') ~ C . There are several cases we need to consider. 

1. s ^ Lpc and s ^ Lpc- 



Then 



Pro6,.,(C,C',/,s) 



E 



7r(s)(A,/i') 



{X,fJ,')£Supp{TT{s)) 




A-e 



tGSupp{ij,') 
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thus there exists s -^p fi such that 

Proh^AC, C\I, s) 

rb 

= / A-e-^"^- ^ ii{t)- Proh.,AC,C',lQx,s^{x,t))dx. 
° tesuppip) 

By induction if s ^ ipc and s y= (pc' , then r ^ ipc' and either r [= ipc or 
r ^ ipc, the case when r ^ ipc is trivial, since Prober' ,r{C,C , I jf) — for 
all tt'. Suppose that r ^ lySc" and r ^ (/Sp, since s -< r, there exists r — >p 
such that fi in other words, v{C) < /i(C) for each -< downward 

closed set C, hence there exists tt' such that 



/ A-e"^^- ^ v{t) ■ Prob^>AC,C\I ex,r^ix,t))dx 

° tl^Supp{u) 
nb 

< A-e"^^- ^ /^(t) • Fro6^,t(C,C",/ea;,s-(a;,t))dx 
" teSuppip) 

by induction. By definition there exists {vi}i<i<n and {pi}i<i<n such that 
Y^i<i<nPt = 1 *^iid J2i<i<nP^ -Vi^v. Let tt' choose transition (A, i/^) with 
probability at state r, then it is not hard to see that 

/ A-e"^^- i^{t) ■ Prob^,AC,C',lQx,r^{x,t))dx 

° t&Supp(v) 

J2 ^'M(A,^') 

{Xy)&Supp{TT'{r)) 

I A-e"^''- XI • ■Pro&7r',t(C,C",/ea;,r-(a;,t))da; I , 

■'^ tesupp(u') J 

thus there exists tt' such that ProbTji C' , /, r) < _Pro6^ ^(C, C", /, s). 
2. s ^ lySc" and s ^ (pc"- 
Then 

Prob^,siC,C\I,s) = e-^''+ ^ ^(s)(A,/^') 

(A,/i')ESupp(7r(s)) 

/ A-e"-^^- X /^'(i) •i'ro&^,t(C,C",/ex,s-(a;,<))da; j , 

° t£Supp(fj,') J 

and there exists s /i such that 
Prob^,,iC,C',I, s) = 6-^'^ 
+ / A-e"^^- X n{t) ■ Prob^,tiC,C',I ex,s^{x,t))dx. 
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By induction there arc four cases: either r 1= ipc and r |= ipc , r ^ ipc 
and r |= (pc, H and r 'Pc, or r ^ (/Sc and r ^ iy9c/. The first 
case is similar with Clause 1, and is omitted here. If r ^ ipc and r \= 
ipc'i then ProbT^^s[C,C' ,I,s) = Probn^riCjC ,I,r) = 1 if a = 0, otherwise 
ProbTr',r{C,C' ,I,r) = 0, thus such tt' always exists. When r \= (pc and 
r y= Lpc' , there exists r — )-p v such that 



° teSupp{u) 

+ A-e"^^- X z/(t) •Pro&^.,t(C,C",/ex,r-(a;,0)rf: 



t^Supp{u) 



+ f A-e~^^- V iy{t) ■ Prob.^>,t{C,C' ,1 e x,r^{x,t))da 



< I X-e-^^'dx 



oo 



/ ^•e"^''- v{t)-Prob^,^t{C,C',Iex,r^{x,t))dx 

° t^Supp{v) 
pa 

/ A-e"^"^- X 1^(0 • Pro6^^t(C,C",/ea;,r-(x,t))da 

° t^Supp{v) 

<e-^"'+ f A-e"^^- V //(t) • Pro6^,t(C,C",/ex,s-(x,t))da; 



Let tt' be a scheduler which chooses transition (A, i^i) with probability pi, 
then ProbTr'^r{C,C' ,I,r) < ProbTr^s{C,C' , I, s). The last case is trivial since 
Prob^^riC, C", /, r) = for aU n. 
3. The other cases are trivial. 

In all cases wc have proved that for each tt, C, C , and /, there always 
exists tt' such that Prob.„f^r{C,C' ,I,r) < Prob.^^s{C,C' ,I,s). Suppose that r 1= 
P>g((/3i U ip2), that is, Prob-^>^riSat((pi), Sat{(p2), I,r) > q for all tt'. If s ^ 
P>q{fi U </32) which means there exists tt such that 

Prob-„,s{Sat{Lpi),Sat{ip2),I,s) < q, 

then there docs not exist tt' such that 

ProbTr',riSat{Lpi), Sat{ip2),I,r) < Prob.„^siSat{Lpi), Sat{tp2), I , s) 
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which contradicts the assumption that s ^ r, hence s |= P>q{(pi (p2)- 

Since uniformization does not change the satisfaction of CSL\x, thus the proof 
of Clause 3) and 4) is straightforward according to Definition 11. 

C.2 Proof of Theorem 8 

Proof. 1. Let TZ = {{s\\t,r \\t) \ s -< r}, it is enough to show that 7^ is a strong 
simulation. Suppose that {s \ \ t) TZ {r \ \ t) , and s\\t ^ By Definition 4 there 
exists s — ^ and t — ^ v such that A = Ai + A2, /i = ^ ■ (/ii || 2?t) + ^ • 

(2?s III')- Since s ^ there exists r -—^p /i'^ such that /ii C-r^ yu'j^, thus 
{m II T>t) Qn (m'i II ^t)j as well as {Vs \ \ v) [T>r Wv) by induction. As 

a result there exists r 1 1 1 ^' = ^- ■ {^'i\\Vt) + ^ ■ {Vr 1 1 1^), so /i C7J ^' 
which completes the proof. 

2. Suppose that s ^ r, then according to Definition 11, s < f. Due to 
Theorem 2, we have s\\t -< f ||f. As a result s||i -< r\\t, therefore 
s||i ^ r||t. 

3. The proof is -< = ~ is directly from Definition 11. Since uniformization 
preserves CSL^^^Xi thus CSL^ = CSL^^x in a uniformized CTMDP. 

4. The proof is straightforward based on Clause 2) and Theorem 7. 

5. The proof of~ C (-<n-<^^)is trivial and omitted here. To show that 

n -=;~^) is strictly coarser than ~, it is enough to give a counterexample. 
Suppose we have three states si, S2, and S3 such that si -< S2 ^ S3 but 
S3 'A S2 7^ si- Let s and r be two states such that L(s) ~ L{r). In addition 
s has three transitions: s , s Vg^ , s Vg^ , and r only has two 

transitions: s T^si, s T^s^ - Then it should be easy to check that s ^ r 
and r < s, the only non-trivial case is when s Since S2 -< S3, thus 

there exists r \ such that D^.^ T^ss - But obviously s ^ r, since 

the transition s \ Vg^ cannot be simulated by any transition of r. 

6. The counterexample adopted in the proof of Clause 3) in Theorem 8 also 
applies here, thus the proof is similar and omitted. 

C.3 Proof of Theorem 9 

Proof. We first show that is a preorder. The reflexivity is trivial and we only 
show the proof of transitivity. Suppose that s -<i t and t -<i r, we need to prove 
that s -<i r. By Definition 12 there exists two strong i-depth simulation such 
that s 72.1 i and t 7^2 J'- Let 7?. = 7^io7?.2 = {(si, S3) I Els2.(si S2AS2 S3)}, 
it is enough to show that 72. is a strong i-depth simulation. Similar with the proof 
of Lemma 3 it can be shown that TZi U 7^2 C 72, thus for each TZ downward 
closed set C, it is also 72i and 7^2 downward closed. The following proof is 
straightforward, and is omitted here. 

To prove that '^csl- ^ it is enough to show that TZ ~ {(s, r) | 

s '^csL~ ^} a strong i-depth simulation. By definition given a TZ downward 
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closed set C, there exists Lpc such that Sat{ipc) ~ C. Suppose that s TZ r 
and for two TZ downward closed sets C, C" and / C [0, oo), there exists tt such 
that ProbTr\r{C,C' ,i,I,r) > Prob.,r,s{C,C' ,i, I , s) for any tt'. Since Pr7r,s('0) = 
Prob-^^siCjC I , s) where ip = Vc^lfCj thus there exists q such that r \= 
P>qip but s ^ Pyqip which contradicts with the assumption that s ^^^sl^ ''j 
so there must exist tt' such that ProbTr' .r{C, C , i, I, r) < Probj^ siC, C", i, /, s). 

To show that -<i C "^csl" ' need to prove that if s -<i r, then 
r \= ip implies s \= ip for any ip of CSL~. We only consider the case when 
p = P>q{ipi U,[iy92) since all the other operators are either similar or trivial. 
Suppose that r |= p^ in the other words, Prj^.sip') > <Z for any scheduler tt. 
Let C = {s £ 5 I s 1= pi} and C — {s £ S \ s ^ 'P2}, it is obvious that C 
and C are -<i downward closed by induction. Then Prob.^^r{C,C' ,i,I,r) > q 
for any scheduler tt. Assume that s ^ p, that is, there exists tt' such that 
ProbTr',s{C,C' ,i, I, s) < q. By definition of ^i, there should exist tt such that 
ProbTr^r{C,C' ,i, I,r) < Prob^' ,s{C,C' ,i, I, s) < q which contradicts with the 
fact that r \= p, thus s \= p. 

Since in a finite system we only have finite equivalence classes, thus the same 
argument applied in Theorem 5 also works here. 

Similar as the proof of Clause 3 of Theorem 5, Example 8 can be used as a 
counterexample here too, thus -<i with i > 1 is not congruent in general. 

Now we prove that there exists n such that ^„ = ^csl\x- We first 
shows that s ^„ r implies that s -<csLyx ^ i-*^- — ^csl^x- Since 

s ^, then s -<n f, thus s ^csl^x ^ shown before. Since uniformization 
does not change the satisfaction of CSL\x, therefore s ^csl^x To show that 
"^cSLyx — prove that s -<csLyx ^ implies that s ^„ r. It is easy to 

see that ^csl^x ~ "^csl in a uniformized CTMDP, thus s ^csl^x ^ implies 
that s -<csL ^- Therefore s ^ r i.e. s ^ r. 

We prove that ^1 is congruent. By Definition 12, s ^1 r iff s -<i f, so we 
only need to show that ^1 is congruent in uniformized CTMDPs. It is enough 
to show that 7^ = {(s || r || t) | s -<i r} is a strong 1-step simulation. Note 
that in a uniformized CTMDP, we can change the definition of strong 1-step 
simulation as follows: s TZ r implies that for any TZ download closed set C and 
s ^ fi such that /i(C) > 0, there exists r — > 1/ such that i'(C) < n{C). Suppose 
that for a TZ download closed set C, and s\\t —i' fi with /i(C) > 0, there exists 
s ^ Hi and t ^ ^2 such that ^ • {hi \ \ Vt) + \ ■ (T^s \ \ M2) — M> thus there exists 
TZ downward closed sets Ci and C2 such that ^ ■ /ii(Ci) + i • /^2(C2) = m(C) 
where {{s'\\t \ s' e Ci} U {r \ \t' \ t' \ C2}) C C. Since s -<i r, there exists 
r — )■ i/i such that !^i(Ci) < /xi(Ci), by induction there exists r ^ v such that 
i-A<i(Ci) + i-A*2(C2) < \ -vi{Ci) + \-H2{C2) i.e. v{C) < ii{C). This completes 
our proof. 

C.4 Proof of Theorem 10 

Proof. 1. According to Definition 12, — in a uniformized CTMDP. 
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2. The proof of C (^j n ) is trivial. Note that the counterexample 
used in Clause 5) of Theorem 8 also applies here, thus C {~<i C] ~<l^)- 

3. Similar as Clause 2), the proof of «i C n ~~^) is trivial, and moreover 
the counterexample used in Clause 5) of Theorem 8 also applies here, thus 

C. 5 Proof of Lemma 5 

Proof. First we can show that -<(;5l- = ^csl" • "^^^ proof is similar with 
the proof of ^^sl^ = ^csl" Lemma 4, and is omitted here. Therefore 
« n ^cSL" coincides with '^cSL^Un • 

D Proofs of Section 7 

D. l Proof of Lemma 8 

Proof. The proof of ^ = ~ctmc is trivial, since in a CTMC there is only one 
transition for each state, thus we can simply replace — >p with — >. The condition 
Xs ■ IJ-s{C) = A,. • /ir(C) for each C coincides with the condition: i) = A^, and 

ii) ^.s U fir . 

We first prove that ~ implies ~ctmc- Let TZ = \ s ~ r} is a weak 

bisimulation referring to Definition 13. Suppose that s /is, we need to prove 

that r fir such that A^ ■ HsiC) = A^ • Hr{C) for all C £ S/TZ with C ^ [s]-jz = 

[r]fi. According to Definition 6, s w r if s ~ f. By Definition 2, if s fis, then 

s ^ jjL such that /i = ■'Ds + ^ -jig where jig is defined as expected. Therefore 

there exists f v such that fi ^ v where v = ■ T^f + ^ ' Pr- Obviously 

if there exists C G S/TZ with C ^ [s]-ji = [r]-ji such that A,, ■ fis{C) ^ Xr ■ f^r{C), 
then fi{C) 7^ I'iC) since /i(C') = ■ fJ-s{C) and i'(C') = ^ ■ fir{C), thus it is 
impossible for /i ~ z^. 

To show that «ctmc implies ~, it is enough to show that 7Z = {(s,'') | 
s ~CTMC ^} is a weak bisimulation according to Definition 6, that is, we need 
show that TZ = {{s,r) \ s «ctmc r} is a strong bisimulation by Definition 5. 

Suppose that s ^ fi, then there exists s fi^ such that fi = -^^g^ •X'^ + ^ ■ /I,. 

Since s ~ctmc r, there exists r /i,. such that As • fJ-siC) ~ A,- • fir{C) for all 

equivalence class C ^ [s]«ctmc ~ ['"]~ctmc- Therefore there exists f ^ v such that 
u = ^^^-Vf + ^-flr and /x(C') = i'(C') for all equivalence class C ^ [s]n = [r]TZ: 
since /i(C') = ^ • /is(C) and z^(C') = ^ • fir{C) i.e. fiTZ v. 

D.2 Proof of Lemma 9 

Proof. According to Definition 11 and 14, the only difference between -< and 
^CTMC is that s ^ r requires that As = A^ while s -<ctmc only requires that 
As < Xr, thus -< C -<cTMC- In a uniformized CTMC, As = A^ for any s and r, 

thus -< = -<CTMC = ^- 
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D.3 Bisimulations for Probabilistic Automata 

In this section we recall some definitions introduced in [25] and [27]. First we 
give the definition for probabilistic automata: 

Definition 15. A tuple Ai = (5*, — AP, L, sq) is a probabilistic automata (PA) 
if S is a finite but non-empty set of states, — i-C S x Dist(S) is a transition 
relation, AP is a finite set of atomic propositions, L : S ^ 2^^ is labeling 
function, and sq (z S is the initial state. 

Below follows the definitions of strong probabilistic bisimulation and simula- 
tion. 

Definition 16. Let Ai = {S, AP, L, sq) be a PA. An equivalence relation 
TZ Q S xS is a strong probabilistic bisimulation iff s TZ r implies that L{s) = L{r) 
and for each A n, there exists a combined transition /i' such that /i TZ fx' . 

We write s '^p r whenever there is a strong probabilistic bisimulation TZ 
such that s TZ r. 

Definition 17. A relation TZ <^ S x S is a strong probabilistic simulation iff 
s TZ r implies that L(s) ~ Lir) and for each /i, there exists a combined 
transition -^p ji' such that fj, n' . 

We write s -<p r whenever there is a strong probabilistic simulation TZ such 
that s TZ r. 

In order to define strong i-depth branching bisimulation, we define Prob„^s{C, C", 
which denotes the probability from s to states in C via states in C possibly 
in at most n steps under scheduler cr, where uj is used to keep track of the 
path and only deterministic schedulers are considered in the following. Formally, 
Proba^s{C, C", n, w) equals 1 if s e C", and else if7i>0A(seC \ C"), then 

Proha,a{C,C' ,n,uj) = fi {r) ■ Proha,r{C,C' ,n — l,ior). 

r(^supp(fj,' ) 

where a{uj){s, fi') = 1, otherwise equals 0. 

Below follows the definition of strong z-depth branching bisimulation where 
s r iff Lis) = L{r). 

Definition 18. Let M = {S, ^, AP, L, sq) be a PA. A relation TZ C S x S is a 
strong i-depth branching bisimulation if i > I and s TZ r implies that s r 
and for any TZ downward closed sets C, C , 

1. if Proba^s{G, C ,i,s) > for a scheduler a , then there exists a scheduler a' 
such that Prob^i ^r{C, C", i, r) < Prob„^s{C, C", i, s), 

2. if Probe _r{C,C' ,i,r) > for a scheduler a, then there exists a scheduler a' 
such that Prober' ^s{C, C ,i, s) < Probij^r{C,C' ,i,r). 

We write s ^\ r whenever there is a strong i-depth branching bisimulation TZ 
such that s TZ r. 
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Lets -<o r iff L(s) = L(r), then 

Definition 19. A relation TZ ^ S x S is a strong i-depth branching simulation 
with i > 1 iff s TZ r implies that s ^^^i r and for any TZ downward closed sets 

C, C , whenever Prob^_s{C, C", i) > for a scheduler a, there exists a' such that 

Prob^,^r{C,C',i) < Prob^^,{C,C',i). 

We write s -<\ r whenever there is a strong i-depth branching simulation TZ 
such that s TZ r. 

Note the definition of is slightly different from [27] where we let 
Proba'AC,C',i) > Prob„AC,C',i) 

instead of Proba-'.r{C,C' ,i) < Probcr^C^C ,i) in Definition 19. The choice be- 
tween < and > is not arbitrary, but depends on the relative operator in the 
state formula T'>p{ip) of CSL^. Intuitively, if we use > p in the logic, we actually 
care about whether the minimal probability of the paths satisfying "0 is equal or 
greater than p or not, while if we use < p instead, then we need to check whether 
the maximal probability of the paths satisfying ip is equal or less than p or not. 
In the contrary, if we use > in Definition 19, we require that the maximal prob- 
ability, max{Proba-.r{C,C' ,i)}, should not be less than max{Proba,siC,C' ,i)} 
for any C and C". Likewise, if we use <, wc require that the minimal probability, 
mm{Probcr,r{C,C' ,i)}, should not be greater than mm{Probc^s{C,C' ,i)}. 

D. 4 Weak Simulation of CTMC 

We recall the definition of weak simulation on CTMC introduced in [2]. Let 
Post{s) = Supp{fis) denote the successors of s. Bellow follows the definition of 
weak simulation where i = 1,2: 

Definition 20. Given a CTMC, let TZ C S x S be a weak simulation iff for 
Si TZ 32'. L{si) ~ L{s2) and there exists functions rji : S ^ [0, 1] and sets Ui, Vi C 
S where Ui — {ui G Post{si) \ r/i{ui) > 0} and Vi = {vi G Post{si) \ r/i{vi) < 1} 
such that: 

1. vi TZ S2 for all vi GVi, and si TZ V2 for all V2 S V2. 

2. There exists a function A : S x S [0,1] such that: 

(a) A{ui,U2) > implies Ui G Ui and ui TZ U2. 

(b) IfK, > 0, then Ki-J2u^^u2 ^(^^'"2) = miw)-fisAw) and K2-J2uieUi ^("1 
r/2{w) ■ pLs^{w) for all states w € S where Ki = J2u Vii'^i) ' l^si(ui). 

(c) J2meUi Viui) ■ • fJ-sAui) < E„2ec/2^(^2) • A^, ■ HsAu2)- 

Si is weakly simulated by S2, written as si ^ctmc S2> iff there exists a weak 
sim.ulation TZ such that si TZ S2- 
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Example 11. Consider states sq and rg in Example 10 and shown in Fig. 4. We 
first show that sq ~csl" ^o- It is easy to check that vi ~csl'' ^o, thus 
the transition from sq to vi is invisible. For the transition from sq to ui, vq can 
perform exactly the same transition, thus no formula ip of CSL^^ exists such 
that Sq \== if but ro ^ ip. Secondly, we show that sq ^ctmc fQ. Obviously 
vi ^CTMC ro, but ui ^CTMC ^0, So ^CTMC ri, and sq ^ctmc "i because 
So, fi, and ui have different labels. Thus the only possible partition is letting 
Ui = {ui}, Vi = {vi}, and U2 = {ri,ui}, V2 = 9 i.e. = 0, Vii^i) = 1, 

V2{ri) = V2{ui) = 1. According to Definition 20 Ki ~ 0.5 and K2 = 1. Since 
ui ^CTMC ri, thus Z\(ui,ri) = 0, but then K2 ■ A{ui,ri) = 7^ 0.5 = ri2{ri) ■ 
^iroiri) which contradicts the condition of Definition 20, thus sq ^ctmc ''o- 



